A twinning bare bones particle swarm optimization algorithm

A twinning bare bones particle swarm optimization(TBBPSO) algorithm is proposed in this paper. The TBBPSO is combined by two operators, the twins grouping operator (TGO) and the merger operator (MO). The TGO aims at the reorganization of the particle swarm. Two particles will form as a twin and influence each other in subsequent iterations. In a twin, one particle is designed to do the global search while the other one is designed to do the local search. The MO aims at merging the twins and enhancing the search ability of the main group. Two operators work together to enhance the local minimum escaping ability of proposed methods. In addition, no parameter adjustment is needed in TBBPSO, which means TBBPSO can solve different types of optimization problems without previous information or parameter adjustment. In the benchmark functions test, the CEC2014 benchmark functions are used. Experimental results prove that proposed methods can present high precision results for various types of optimization problems.


Introduction
The particle swarm optimizer (PSO) has evolved into a group of algorithms since it was first introduced by Kennedy and Eberhard in 1995. The PSOs belong to the classical evolutionary algorithm, which is inspired by the behavior of birds flocking and fish schooling. Based on the PSOs, more and more swarm evolutionary algorithms are proposed, like as artificial flora algorithm [1], artificial bee colony algorithm [2], fish swarm algorithm [3], firefly algorithm [4], cuckoo search algorithm [5]. The inspiration for these algorithms comes from the migration and reproduction processes of the swarm. In general, the PSO algorithms specialize in solving nonlinear non-stationary problems and have been widely used in parallel computing [6], pattern recognition [7], automatic control [8], transportation engineering [9] and other fields. For example, Shen [10] designs an effective gas cyclone method by using a hybrid PSO and differential evolution algorithm, and shows that the method with higher efficiency and low cost. Sung [11] proposes a method to increase the accuracy of user positioning in indoor environments using wireless-fidelity (Wi-Fi). The core point lies in the PSO algorithm and the selection of its initial weight, and the experiments have demonstrated that the method can achieve a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 higher positioning accuracy. Li [12] presents a novel PSO-based method for hybrid wind turbine towers which is able to reduce the direct investment, labor cost, mechanical cost, and maintenance cost.
Many interesting variants of the standard PSO algorithm have been derived, like the comprehensive learning particle swarm optimizer (CLPSO) [13], the cooperative particle swarm optimizer (CPSO) [14], the hierarchical particle swarm optimizer (HPSO) [15], the unified particle swarm optimizer (UPSO) [16], and the bare bones particle swarm optimizer (BBPSO) [17]. Among them, the BBPSO algorithm as one of its typical mutated versions has also attracted a lot of attention and has been successfully applied to power systems, chemical testing, disease diagnosis, etc. For instance, Zhang [18] presents a hybrid improved BBPSO algorithm to solve dynamic economic dispatch problems. Zhang [19] proposes two evolutionary BBPSO-based feature optimization. Overall, the BBPSO algorithm is simple and easy to implement, just like other swarm algorithms, to find high-precision results for benchmark functions.
The PSO algorithm and its variants have performed relatively well in solving different types of practical problems. However, many researchers still work to overcome the defects of the PSO algorithm like the curse of dimensionality [14], the collapse of the swarm, the premature convergence, the complex multimodal problems, and so on [15]. Meanwhile, in academia and industry areas, a huge number of complex real-world problems are waiting to be solved. Therefore, the variant of PSO algorithms with superior performance is always needed.

Related works
Researchers try to use different evolutionary method like [20][21][22][23]. By reviewing the literature, we can summarize that there are roughly three types of variants of PSO algorithms. The first type of variant focus on the initial weights. Since the original PSO algorithm did not have an inertia weight, Shi [24] proposed a modified PSO algorithm by adding an inertia weight and achieving a faster convergence. Subsequently, a constriction factor was added to the convergence behavior of the PSO algorithm [25]. Some references [15,26] confirmed that this improved approach has effective results on the large collection problems. In the course of engineering practice, the inertia constant is typically taken to be 0.9 [27].
The second type of variant is concerned with the optimal performance of the particles. Angeline [28] proposed an entirely different approach by introducing a form of selection mechanism that allows some good particles to replace some less effective ones. The multiple neighborhoods are applied to the particle population and each neighborhood keeps its own local best solution [29]. This method is not prone to get trapped in local minima but usually has a slow convergence rate. To address this drawback, cooperative methods were introduced into distributed architectures [14]. In [30], Rui and James argued that each individual is influenced not only by the best performer among its neighbors but also by the success of all its neighbors, based on which they designed a fully informed particle swarm optimizer and confirmed its effectiveness using a benchmark function.
The third type of variant centers on making the PSO algorithm simpler. BBPSO is probably the simplest of all PSO variants and has the potential to solve single-objective unconstrained optimization problems. Details can be found in Eq 1 and Algorithm 1.
where the pbestðx t i Þ is the personal best position of the particle i in (t)th generation, Gbest t is the personal best position of the global best particle in (t)th generation, x tþ1 i is the candidate new position for particle i in (t + 1)th generation, GD(α, β) is the Gaussian distribution with a mean α and a standard deviation β. However, it has the disadvantage of being close to collapse. In [31], Tim proved that a collapse-free condition can be obtained by including the motion of the informant, i.e., allowing a small random search over the entire search space at any stage of the optimization. Zhang [18] designed an adaptive interference factor and a new genetic operator was incorporated into the improved BBPSO. Experimental results showed that the improved method has enhanced its searchability. Mauroas [32] proposed a new method that the positions of the particles are chosen from a multivariate t-distribution and obey the rules adapted to their scale matrix. In [33], Li proposed a new BBPSO-based method, in which the behavior of the particles should obey the principle of the first-order difference equation.
Our team has been focused on improving BBPSO. In the beginning, we design BBPSObased method that two particle work in pair [34]. In this algorithm, two particles form as a pair and are placed in different sets of evolutionary strategies. Then, we combined a local search strategy with the BBPSO in [35]. In the same way, we use a dynamic allocation strategy to enhance the search ability of BBPSO in [36]. In recent research work, we have adopted a fission-fusion strategy aimed at partitioning the search space [37]. Base on it, we proposed a fission-fusion hybrid bare-bone particle swarm optimizer (FHBBPSO) algorithm, which combines the fission strategy and the fusion strategy, and the particles are assigned to different local groups to sample the corresponding regions.

Proposal of the twinning bare bones particle swarm optimization algorithm
In this paper, only minimum problems will be discussed, hence a better position in this paper stands for a position with a smaller fitness value. When several positions are discussed, a position with the smallest fitness value is defined as the best position. The particle with a better position is defined as a better particle. The twinning bare bones particle swarm optimization algorithm (TBBPSO) is proposed in this section. In TBBPSO, two particles will form a twin and perform collaborative computing across iterations. The TBBPSO is combined by two main operators, a twins grouping operator (TGO) and a merger operator (MO). The TGO aims at dividing the particle swarm into several sub-groups and the merger operator aims at merging the sub-groups. Particles search around the global best particle and their team leaders in different generations.

The twins grouping operator
The TGO is used to divide the particle swarm into several twins. Each twin contains two particles. Inside a twin, the particle with a smaller fitness value will be pointed as the main particle, and the other one will be the side particle. The next position of a main particle is selected by Eq 2.
where the pbest(main t ) is the personal best position of the main particle in (t)th generation, Gbest t is the personal best position of the global best particle in (t)th generation, pbest(main t+1 ) is the new position for the ZZmain particle in (t + 1)th generation, GD(γ, δ) is the Gaussian distribution with a mean γ and a standard deviation δ, F is the target test function. The side particle of a twin is designed to search around the main particle, hence the next position of a team member is selected by Eq 3.
where the pbest(side t ) is the personal best position of the side particle in (t)th generation, the pbest(main t ) is the personal best position of the main particle in (t)th generation, pbest(side t+1 ) is the new position for the side particle in (t+ 1)th generation, GD(θ, φ) is the Gaussian distribution with a mean θ and a standard deviation φ, F is the target test function. The pseudo code of the TGO is described in Algorithm 2. Algorithm 2 TBBPSO-Grouping 3: Calculate the the Pbest_position, personal best position of particles 4: Record the Gbest_position, global best position of the swarm 5: Record the Gbest_value, global best value of the swarm 6: NLG = 0, NLG is short for number of local group 7: while X 6 ¼ ⌀ do 8: Take two particles x i and x j out of X 9: x i and x j are defied as a twin 10: if Pbest_value(i) < Pbest_value(j) then 11: Chose a NewPosition for x i by Eq 2 12: Chose a NewPosition for x j by Eq 3 13: Chose a NewPosition for x i by Eq 3 16: Chose a NewPosition for x j by Eq 2 17: if New positions of any particles out of R then 19: Chose a new randon position for them in R 20:

Merger operator
The merger operator (MO) is proposed in this section. After grouping, particles were gathered into several local groups including one main local group (MLG) and several sub-local groups (SLGs). In each iteration, the MLS will merge one SLG until no SLG exits. During this process, particles will keep playing as the team leader and the team member. In the MLG, the best particle will be the team leader and others will be members. For all particles, leaders and teammates evolve with the same equations from TGO. The pseudo-code of the MO is given in Algorithm 3. Find the main particles in the MLG 4:

Algorithm 3 TBBPSO-Merger
Other particles in the MLG are side particles 5: In the MLG 6: Chose a NewPosition for x main by Eq 2 7: Chose a NewPosition for x side by Eq 3 8: In each SLG 9: Chose NewPosition for x main by Eq 2 10: Chose NewPosition for x side by Eq 3 11: NLG = NLG − 1 12: end while 13: for i = 1, i � n do 14: if F(NewPosition(i)) < Pbest_value(i) then 15:

Complete process of TBBPSO
To describe the TBBPSO more clearly, the schematic diagram of TBBPSO is given in Fig 1,

Experimental methods
To verify the optimization ability of the TBBPSO, the CEC 2014 benchmark functions (CEC2014BF) [38] are used in the experiments. Details of the CEC2014BF can be found in Table 1. The, BBPSO [17], PBBPSO [34] and DLS-BBPSO [35] are selected into the control group. To make a fair competition, all algorithms use the same population size, iteration times, and best parameters in their original papers. All tests are repeated 31 times and the average results are recorded to reduce accidental errors,. The population size for all algorithms is 100, dimension is 50, max generation time is 1.000E+4. The Measurement Error (ME) is defined as |final gbest value − Theoretically optimal|.

Experiments with top BBPSO-based method
The FHBBPSO [37], one of the top BBPSO-based methods is selected to compare with TBBPSO. These two methods run on the Rotaten High Conditioned Elliptic Function 31 times, the population size of the two methods is 100, dimension is 50, max iteration times is 1000, and the mean MEs are shown in Table 2. It can be seen that at 100 and 200 iterations, the results of the two methods are very close. After one 1000 iterations, the result of TBBPSO is 22.16% smaller than that of FHBBPSO. It can be assumed that TBBPSO has a clear advantage in this set of experiments.

Experimental results with complete set of benchmark functions
MEs are displayed in Tables 3 and 4. In 30 benchmark functions, TBBPSO ranked first in 12 functions and ranked second in 12 functions. In each benchmark function, the algorithm with

PLOS ONE
the first rank will get 1 point, the second rank will get 2 points, the third rank will get 3 points and the fourth rank will get 4 points. The average rank point of TBBPSO is 1.900, the best of the four optimization algorithms. The Friedman statistic test [22] is also implemented and average rank results are shown in Table 4. Specifically, numerical analyses are listed below: • In f 1 , TBBPSO gains the first rank, the results of TBBBOS are 12.88% better than results from DLS-BBPSO, the second-best algorithm.
• In f 2 , TBBPSO gains the second rank, the results of TBBBOS are 44.87% worse than results from PBBPSO, the best algorithm.
• In f 3 , TBBPSO gains the second rank, the results of TBBBOS are 86.22% worse than results from BBPSO, the best algorithm.
• In f 4 , TBBPSO gains the fourth rank, the results of TBBBOS are 25.40% worse than results from BBPSO, the best algorithm.
• In f 5 , TBBPSO gains the first rank, the results of TBBBOS are 0.04% better than results from DLS-BBPSO, the second-best algorithm.
• In f 6 , TBBPSO gains the second rank, the results of TBBBOS are 9.68% worse than results from BBPSO, the best algorithm.
• In f 7 , TBBPSO gains the second rank, the results of TBBBOS are 71.43% worse than results from DLS-BBPSO, the best algorithm.
• In f 8 , TBBPSO gains the first rank, the results of TBBBOS are 1.01% better than results from PBBPSO, the second-best algorithm.
• In f 9 , TBBPSO gains the second rank, the results of TBBBOS are 8.63% worse than results from DLS-BBPSO, the best algorithm.
• In f 10 , TBBPSO gains the third rank, the results of TBBBOS are 12.87% worse than results from PBBPSO, the best algorithm.
• In f 11 , TBBPSO gains the first rank, the results of TBBBOS are 2.70% better than results from BBPSO, the second-best algorithm.
• In f 12 , TBBPSO gains the first rank, the results of TBBBOS are 15.77% better than results from BBPSO, the second-best algorithm.
• In f 13 , TBBPSO gains the first rank, the results of TBBBOS are 8.03% better than results from DLS-BBPSO, the second-best algorithm.
• In f 14 , TBBPSO gains the first rank, the results of TBBBOS are 19.11% better than results from BBPSO, the second-best algorithm.
• In f 15 , TBBPSO gains the second rank, the results of TBBBOS are 0.68% worse than results from DLS-BBPSO, the second-best algorithm.

PLOS ONE
• In f 16 , TBBPSO gains the third rank, the results of TBBBOS are 3.13% worse than results from BBPSO, the best algorithm.
• In f 17 , TBBPSO gains the first rank, the results of TBBBOS are 13.63% better than results from PBBPSO, the second-best algorithm.
• In f 18 , TBBPSO gains the fourth rank, the results of TBBBOS are 18.13% worse than results from BBPSO, the best algorithm.
• In f 19 , TBBPSO gains the second rank, the results of TBBBOS are 5.39% worse than results from DLS-BBPSO, the best algorithm.
• In f 20 , TBBPSO gains the second rank, the results of TBBBOS are 43.42% worse than results from BBPSO, the best algorithm.
• In f 21 , TBBPSO gains the first rank, the results of TBBBOS are 13.02% better than results from BBPSO, the second-best algorithm.
• In f 22 , TBBPSO gains the first rank, the results of TBBBOS are 3.78% better than results from DLS-BBPSO, the second-best algorithm.
• In f 23 , four algorithms give the same results, which means 4 algorithms are trapped by the local minimal and unable to escape.
• In f 24 , TBBPSO gains the fourth rank, the results of TBBBOS are 1.18% worse than results from PBBPSO, the best algorithm.
• In f 25 , TBBPSO gains the first rank, the results of TBBBOS are 0.06% better than results from PBBPSO, the second-best algorithm.
• In f 26 , TBBPSO gains the second rank, the results of TBBBOS are 0.01% worse than results from BBPSO, the best algorithm.
• In f 27 , TBBPSO gains the third rank, the results of TBBBOS are 13.05% worse than results from BBPSO, the best algorithm.
• In f 28 , TBBPSO gains the second rank, the results of TBBBOS are 0.587% worse than results from DLS-BBPSO, the best algorithm.
• In f 29 , TBBPSO gains the second rank, the results of TBBBOS are 0.32% worse than results from BBPSO, the best algorithm.
• In f 30 , TBBPSO gains the second rank, the results of TBBBOS are 3.61% worse than results from DLS-BBPSO, the best algorithm.
Compared with traditional methods, TBBPSO is more reliable and more robust. The MO and the TGO work together to cross the local minimal and search for high precision results. To perform the convergence situation across iterations, the ME in different iterations for BBPSO, PBBPSO, DLS-BBPSO, and TBBPSO is shown in Figs 3-32. The scale on the vertical axis represents the value of ME. The scale on the horizontal axis represents iteration times, 100 on the horizontal axis represents 10,000 iterations.

Discussion and future works
The main reason that TBBPSO can provide high precision results for test functions is that the TGO divides the original swarm into several local groups. This process increases the diversity of the swarm and enhances exploration. The MO makes the main local group merge one local group per iteration while the rest local groups can keep doing a local search. These processes increase the exploitation.
The overall effectiveness (OE) [21] of TBBPSO and other algorithms in the control group is computed by results in Tables 3 and 4. The OE is calculated by Eq 4.
where N is the number of test functions, L is the number of times the target algorithm loses in the competition. Results pf OE is shown in Table 5. Four algorithms tie in f 23 . It can be seen that TBBPSO has the best performance.

PLOS ONE
In the last iteration, all the particles are concentrated at one point, so it is impossible to find a solution with a better position. How to maintain the diversity of the particle population even after very long iterations will be the main direction of future work. The TGO-MO corporation pattern works well in single-objective optimizations. Another direction of the future work is to apply this work pattern to multi-objective optimization problems.

Conclusions
A TBBPSO is proposed in this paper to solve single-objective optimization problems. The TBBPSO works by re-organizing the structure of the particle swarm. The TGO and MO  PLOS ONE cooperate to prevent the particle swarm from losing diversity too fast. The TBBPSO presents high-precision results when handling CEC 2014 benchmark functions. The main reason is the cooperation of the TGO and the MO. The TGO divides the particle swarm into several calculation units, the twins. Each twin contains two particles playing different roles. The particle with a better personal best position plays as the team leader, the other one plays as the team member. Different evolution strategies apply to different roles. Then one twin will be selected as the main local group (MLG) and the other twins will be sub-local groups (SLGs). This strategy can increase the diversity of the particle swarm and give the swarm more chance to escape from local minimums. In the MO, the MLG will merge one SLG in every iteration. With the increase of the population, the local search capability of the MLG is increased. At the same time, other SLGs will search independently of each other to ensure the swarm keeps the ability to escape from local minimums. When all SLGs are merged by the MLS, all particles are in the same

PLOS ONE
group and the algorithm goes to the TGO. This process is simple and fast, no parameter, congestion, or threshold is needed, and the time complexity is o(n).